Skip to content

Conversation

@MichaelKatsoulis
Copy link
Contributor

Add CloudWatch Logs Subscription Filter Support for CloudTrail

Summary

This PR adds support for processing CloudTrail logs delivered via CloudWatch Logs subscription filters to the awslogsencodingextension. Previously, the extension only supported CloudTrail logs stored in S3.

Changes

Core Implementation

  • New fromCloudWatch() method: Handles CloudTrail logs from CloudWatch Logs subscription filter format
  • Format detection: Uses extractFirstKey() to detect the CloudWatch envelope ("messageType") vs S3 format ("Records")
  • CloudWatch event parsing: Unmarshals individual CloudTrail events from logEvents[].message field (each message contains a single CloudTrail event as an escaped JSON string)
  • Resource attributes: Sets CloudWatch-specific attributes (aws.log.group.names, aws.log.stream.names) in addition to CloudTrail attributes (cloud.provider, cloud.region, cloud.account.id)

Format Differences

S3 Format

{
  "Records": [
    { "eventVersion": "1.11", "eventName": "PutObject", ... },
    { "eventVersion": "1.11", "eventName": "GetObject", ... }
  ]
}

CloudWatch Format

{
  "messageType": "DATA_MESSAGE",
  "logGroup": "/aws/cloudtrail/logs",
  "logStream": "123456789010_CloudTrail_us-east-1",
  "logEvents": [
    {
      "id": "eventId1",
      "timestamp": 1418530010000,
      "message": "{\"eventVersion\":\"1.11\",\"eventName\":\"PutObject\",...}"
    },
    {
      "id": "eventId2",
      "timestamp": 1418530020000,
      "message": "{\"eventVersion\":\"1.11\",\"eventName\":\"GetObject\",...}"
    }
  ]
}

Key difference: In CloudWatch format, each logEvents[].message is an escaped JSON string containing a single CloudTrail event, whereas S3 files contain an array of events.

Configuration Example

receivers:
  awslambda:
    cloudwatch:
      encoding: awslogs_encoding

extensions:
  awslogs_encoding:
    format: cloudtrail

service:
  extensions: [awslogs_encoding]
  pipelines:
    logs:
      receivers: [awslambda]
      exporters: [otlp]

Related Issues

Testing

New Test Cases

  • Valid CloudWatch subscription filter format: Tests the new CloudWatch format parsing with real CloudWatch subscription filter event structure
    • Input: testdata/cloudtrail_log_cw.json - CloudWatch subscription filter message with CloudTrail events
    • Expected output: testdata/cloudtrail_log_cw_expected.yaml - Properly mapped OpenTelemetry logs

Breaking Changes

None. This is a backwards-compatible addition that extends existing functionality.

Comment on lines +167 to +170
// CloudWatch subscription filter format
if firstKey == "messageType" {
return u.fromCloudWatch(bufferedReader)
}
Copy link
Contributor

@Kavindu-Dodan Kavindu-Dodan Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately this won't work with CloudWatch subscription filter.

  • For CloudTrail to S3, we have well-defined Records root key 1, so we can work with firstKey
  • For CloudTrail digest, we have a fallback as there's no root key that helps to detect type 2
  • For CloudWatch subscription filter, there are also multiple root keys 3, so it's not guaranteed to get messageType as firstKey. It could be owner, logGroup and depends on AWS unmarshaling to JSON

So IMO we must fallback in this order,

  • First check firstKey to be Records : This is guaranteed per current contract -> Stream processing of the JSON
  • Then as fallback, perform a json.RawMesage unmarshal
    • Check for known CW key like logGroup , messageType -> This is CW trigger
    • Try unmarshaling as Digest record

Let me know your thoughts :)

Footnotes

  1. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-examples.html

  2. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-validation-digest-file-structure.html

  3. https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#LambdaFunctionExample

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[awslogsencodingextension] Allow CloudTrail logs unmarshaling support for logs received through CloudWatch

3 participants